An Example of Reliability Testing and Establishing Performance Profiles for Non-Parametric Data from Performance Analysis

Recent research has proposed a simple empirical way of examining the variance of the means of performance indicators as they approach their final stability values, as more and more performances are analysed. This research aims to provide practical insight into the issues associated with establishing reliability studies, and also examining whether normative profiles of performance indicators have been established, in performance analysis, by using a practical example from recent a recent analysis project of the World Cup for association football. Ten operators collected data from the 2002 World Cup Finals, recording all performance data that occurred during the duration of a game (including injury time but not extra time) The relevant games were viewed through replaying of videotapes, using a jog shuttle on the video player. Training of each operator took place prior to any data collection. Definitions of specific actions used as points of analysis were listed, discussed and used in the training of the operators. In the use of complex analysis systems such as in this example, the reliability study is more than a way of establishing the boundaries of acceptable error, it can highlight the specific need for more training by certain operators. Complex systems subtend different levels of skill from individual to individual, so it is unreasonable to expect everyone to require the same amounts of training. Further a full reliability study will also demonstrate where the operational definitions of the performance indicators are not clear enough, and further redefinitions may be necessary. It was also concluded that, contrary to previous ideas, if an action had a high frequency of occurrences within performances, then relatively fewer matches would be required to obtain a normative profile of this action. But it is clear that it is the variance of these data from match to match that will determine how many matches are required to reach stability. Further research could investigate statistical methods based on the respective variances to predict the number of matches required, thus replacing these empirical techniques.